{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "59bca259",
   "metadata": {},
   "source": [
    "# 订阅智能体\n",
    "\n",
    "输入`pip show metagpt`查看版本\n",
    "类似：\n",
    "\n",
    "> Version: 0.8.0\n",
    "Summary: The Multi-Agent Framework\n",
    "Home-page: https://github.com/geekan/MetaGPT\n",
    "Author: Alexander Wu\n",
    "Author-email: alexanderwu@deepwisdom.ai\n",
    "License: MIT\n",
    "Location: c:\\users\\liwei\\appdata\\roaming\\python\\python39\\site-packages\n",
    "Requires: aiofiles, aiohttp, aioredis, anthropic, anytree, beautifulsoup4, channels, dashscope, faiss-cpu, fire, gitignore-parser, gitpython, google-generativeai, imap-tools, ipykernel, ipython, ipywidgets, jieba, lancedb, libcst, loguru, meilisearch, nbclient, nbformat, networkx, numpy, openai, openpyxl, pandas, Pillow, playwright, pydantic, python-docx, PyYAML, qdrant-client, qianfan, rank-bm25, rich, scikit-learn, semantic-kernel, setuptools, socksio, ta, tenacity, tiktoken, tqdm, typer, typing-extensions, typing-inspect, websocket-client, websockets, wrapt, zhipuai\n",
    "\n",
    "我们先来完成网页爬取的功能，我们教程直接爬取当天不分国家语言和编程语言的热门仓库进行分析，如果有特殊要求，爬取加上筛选条件条件后网页即可。我们先打开[https://github.com/trending](https://github.com/trending) 网页，观察网页内容，找到我们需要的内容对应的 html 元素，。\n",
    "\n",
    "如果熟悉爬虫的就可以直接写爬取和解析脚本了，如果不熟悉的也没关系，我们可以 用 ChatGPT 辅助开发： 首先我们将trending页面保存到 本地github-trending-raw.html\n",
    "格式化后发现内容非常多，大概600多k，还有一些svg源码，因为一般用CSS足以定位 html里的元素，所以我们可以对html内容进行瘦身，可以使用以下的脚本："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "5c0f348d",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Name: metagpt\n",
      "Version: 0.8.1\n",
      "Summary: The Multi-Agent Framework\n",
      "Home-page: https://github.com/geekan/MetaGPT\n",
      "Author: Alexander Wu\n",
      "Author-email: alexanderwu@deepwisdom.ai\n",
      "License: MIT\n",
      "Location: /usr/local/lib/python3.10/site-packages\n",
      "Requires: aiofiles, aiohttp, aioredis, anthropic, anytree, beautifulsoup4, channels, dashscope, faiss-cpu, fire, gitignore-parser, gitpython, google-generativeai, imap-tools, ipykernel, ipython, ipywidgets, jieba, lancedb, libcst, loguru, meilisearch, nbclient, nbformat, networkx, numpy, openai, openpyxl, pandas, Pillow, playwright, pydantic, python-docx, PyYAML, qdrant-client, qianfan, rank-bm25, rich, scikit-learn, semantic-kernel, setuptools, socksio, ta, tenacity, tiktoken, tqdm, typer, typing-extensions, typing-inspect, websocket-client, websockets, wrapt, zhipuai\n",
      "Required-by: \n"
     ]
    }
   ],
   "source": [
    "!pip show metagpt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "893c1940",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "--2025-02-26 14:48:15--  https://github.com/trending\n",
      "Resolving github.com (github.com)... 20.205.243.166\n",
      "Connecting to github.com (github.com)|20.205.243.166|:443... connected.\n",
      "HTTP request sent, awaiting response... 200 OK\n",
      "Length: unspecified [text/html]\n",
      "Saving to: ‘github-trending-raw.html’\n",
      "\n",
      "github-trending-raw     [  <=>               ] 508.12K  1.18MB/s    in 0.4s    \n",
      "\n",
      "2025-02-26 14:48:17 (1.18 MB/s) - ‘github-trending-raw.html’ saved [520312]\n",
      "\n"
     ]
    }
   ],
   "source": [
    "!wget https://github.com/trending -O github-trending-raw.html"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "570382e7",
   "metadata": {},
   "outputs": [],
   "source": [
    "from bs4 import BeautifulSoup\n",
    "\n",
    "with open(\"github-trending-raw.html\") as f:\n",
    "    html = f.read()\n",
    "\n",
    "soup = BeautifulSoup(html, \"html.parser\")\n",
    "for i in soup.find_all(True):\n",
    "    for name in list(i.attrs):\n",
    "        if i[name] and name not in [\"class\"]:\n",
    "            del i[name]\n",
    "\n",
    "for i in soup.find_all([\"svg\", \"img\", \"video\", \"audio\"]):\n",
    "    i.decompose()\n",
    "\n",
    "with open(\"github-trending-slim.html\", \"w\") as f:\n",
    "    f.write(str(soup))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "80971951",
   "metadata": {},
   "source": [
    "经过以上的脚本处理之后，大概还有100多k。对于爬虫来说，重要的是Html的结构，处理后的Html文件其实有大量的信息是重复的，如果我们要让GPT协助我们写爬虫脚本，只需要截取部分信息就可以了。\n",
    "**接下来解析一下html文件**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "f9421441",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Name: vllm-project /\n",
      "\n",
      "      aibrix\n",
      "URL: https://github.com/vllm-project/aibrix\n",
      "Description: Cost-efficient and pluggable Infrastructure components for GenAI inference\n",
      "Language: Jupyter Notebook\n",
      "Stars: 2,363\n",
      "Forks: 200\n",
      "Today's Stars: 570 stars today\n",
      "\n",
      "Name: al1abb /\n",
      "\n",
      "      invoify\n",
      "URL: https://github.com/al1abb/invoify\n",
      "Description: An invoice generator app built using Next.js, Typescript, and Shadcn\n",
      "Language: TypeScript\n",
      "Stars: 4,157\n",
      "Forks: 411\n",
      "Today's Stars: 875 stars today\n",
      "\n",
      "Name: NirDiamant /\n",
      "\n",
      "      GenAI_Agents\n",
      "URL: https://github.com/NirDiamant/GenAI_Agents\n",
      "Description: This repository provides tutorials and implementations for various Generative AI Agent techniques, from basic to advanced. It serves as a comprehensive guide for building intelligent, interactive AI systems.\n",
      "Language: Jupyter Notebook\n",
      "Stars: 6,969\n",
      "Forks: 933\n",
      "Today's Stars: 466 stars today\n",
      "\n",
      "Name: deepseek-ai /\n",
      "\n",
      "      awesome-deepseek-integration\n",
      "URL: https://github.com/deepseek-ai/awesome-deepseek-integration\n",
      "Description: None\n",
      "Language: None\n",
      "Stars: 22,296\n",
      "Forks: 2,416\n",
      "Today's Stars: 1,089 stars today\n",
      "\n",
      "Name: mishushakov /\n",
      "\n",
      "      llm-scraper\n",
      "URL: https://github.com/mishushakov/llm-scraper\n",
      "Description: Turn any webpage into structured data using LLMs\n",
      "Language: TypeScript\n",
      "Stars: 4,195\n",
      "Forks: 237\n",
      "Today's Stars: 73 stars today\n",
      "\n",
      "Name: drawdb-io /\n",
      "\n",
      "      drawdb\n",
      "URL: https://github.com/drawdb-io/drawdb\n",
      "Description: Free, simple, and intuitive online database diagram editor and SQL generator.\n",
      "Language: JavaScript\n",
      "Stars: 24,502\n",
      "Forks: 1,738\n",
      "Today's Stars: 172 stars today\n",
      "\n",
      "Name: ggerganov /\n",
      "\n",
      "      ggwave\n",
      "URL: https://github.com/ggerganov/ggwave\n",
      "Description: Tiny data-over-sound library\n",
      "Language: C++\n",
      "Stars: 4,257\n",
      "Forks: 243\n",
      "Today's Stars: 900 stars today\n",
      "\n",
      "Name: maybe-finance /\n",
      "\n",
      "      maybe\n",
      "URL: https://github.com/maybe-finance/maybe\n",
      "Description: The OS for your personal finances\n",
      "Language: Ruby\n",
      "Stars: 41,245\n",
      "Forks: 2,921\n",
      "Today's Stars: 453 stars today\n",
      "\n",
      "Name: langgenius /\n",
      "\n",
      "      dify\n",
      "URL: https://github.com/langgenius/dify\n",
      "Description: Dify is an open-source LLM app development platform. Dify's intuitive interface combines AI workflow, RAG pipeline, agent capabilities, model management, observability features and more, letting you quickly go from prototype to production.\n",
      "Language: TypeScript\n",
      "Stars: 73,678\n",
      "Forks: 10,739\n",
      "Today's Stars: 1,256 stars today\n",
      "\n",
      "Name: FreeTubeApp /\n",
      "\n",
      "      FreeTube\n",
      "URL: https://github.com/FreeTubeApp/FreeTube\n",
      "Description: An Open Source YouTube app for privacy\n",
      "Language: JavaScript\n",
      "Stars: 15,860\n",
      "Forks: 997\n",
      "Today's Stars: 400 stars today\n",
      "\n"
     ]
    }
   ],
   "source": [
    "import aiohttp\n",
    "import asyncio\n",
    "from bs4 import BeautifulSoup\n",
    "\n",
    "def fetch_html(url):\n",
    "    with open(url, encoding=\"utf-8\") as f:\n",
    "        html = f.read()\n",
    "    return html\n",
    "\n",
    "async def parse_github_trending(html):\n",
    "    soup = BeautifulSoup(html, 'html.parser')\n",
    "\n",
    "    repositories = []\n",
    "\n",
    "    for article in soup.select('article.Box-row'):\n",
    "        repo_info = {}\n",
    "        \n",
    "        repo_info['name'] = article.select_one('h2 a').text.strip()\n",
    "        repo_info['url'] = article.select_one('h2 a')['href'].strip()\n",
    "\n",
    "        # Description\n",
    "        description_element = article.select_one('p')\n",
    "        repo_info['description'] = description_element.text.strip() if description_element else None\n",
    "\n",
    "        # Language\n",
    "        language_element = article.select_one('span[itemprop=\"programmingLanguage\"]')\n",
    "        repo_info['language'] = language_element.text.strip() if language_element else None\n",
    "\n",
    "        # Stars and Forks\n",
    "        stars_element = article.select('a.Link--muted')[0]\n",
    "        forks_element = article.select('a.Link--muted')[1]\n",
    "        repo_info['stars'] = stars_element.text.strip()\n",
    "        repo_info['forks'] = forks_element.text.strip()\n",
    "\n",
    "        # Today's Stars\n",
    "        today_stars_element = article.select_one('span.d-inline-block.float-sm-right')\n",
    "        repo_info['today_stars'] = today_stars_element.text.strip() if today_stars_element else None\n",
    "\n",
    "        repositories.append(repo_info)\n",
    "\n",
    "    return repositories\n",
    "\n",
    "async def main():\n",
    "    url = 'github-trending-raw.html'\n",
    "    html = fetch_html(url)\n",
    "    repositories = await parse_github_trending(html)\n",
    "\n",
    "    for repo in repositories:\n",
    "        print(f\"Name: {repo['name']}\")\n",
    "        print(f\"URL: https://github.com{repo['url']}\")\n",
    "        print(f\"Description: {repo['description']}\")\n",
    "        print(f\"Language: {repo['language']}\")\n",
    "        print(f\"Stars: {repo['stars']}\")\n",
    "        print(f\"Forks: {repo['forks']}\")\n",
    "        print(f\"Today's Stars: {repo['today_stars']}\")\n",
    "        print()\n",
    "        \n",
    "await main()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c7999a73",
   "metadata": {},
   "source": [
    "上面这些代码的作用是解析一下github-trending的数据。由于github-trending网站打开比较慢。我们可以先把github的trending页面保存到本地，再进行解读。\n",
    "# 继续导入其他相关的库"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "1891af07",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting aiocron\n",
      "  Obtaining dependency information for aiocron from https://files.pythonhosted.org/packages/f2/82/90d3c43e137d06496e0305ce06596a0e36c7cd13c6de986378002c0fd749/aiocron-2.1-py3-none-any.whl.metadata\n",
      "  Downloading aiocron-2.1-py3-none-any.whl.metadata (4.3 kB)\n",
      "Collecting cronsim>=2.6 (from aiocron)\n",
      "  Obtaining dependency information for cronsim>=2.6 from https://files.pythonhosted.org/packages/8c/dd/9c40c4e0f4d3cb6cf52eb335e9cc1fa140c1f3a87146fb6987f465b069da/cronsim-2.6-py3-none-any.whl.metadata\n",
      "  Downloading cronsim-2.6-py3-none-any.whl.metadata (6.9 kB)\n",
      "Collecting python-dateutil>=2.9.0 (from aiocron)\n",
      "  Obtaining dependency information for python-dateutil>=2.9.0 from https://files.pythonhosted.org/packages/ec/57/56b9bcc3c9c6a792fcbaf139543cee77261f3651ca9da0c93f5c1221264b/python_dateutil-2.9.0.post0-py2.py3-none-any.whl.metadata\n",
      "  Downloading python_dateutil-2.9.0.post0-py2.py3-none-any.whl.metadata (8.4 kB)\n",
      "Requirement already satisfied: tzlocal>=5.2 in /usr/local/lib/python3.10/site-packages (from aiocron) (5.2)\n",
      "Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.10/site-packages (from python-dateutil>=2.9.0->aiocron) (1.16.0)\n",
      "Downloading aiocron-2.1-py3-none-any.whl (5.6 kB)\n",
      "Downloading cronsim-2.6-py3-none-any.whl (13 kB)\n",
      "Downloading python_dateutil-2.9.0.post0-py2.py3-none-any.whl (229 kB)\n",
      "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m229.9/229.9 kB\u001b[0m \u001b[31m1.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0mta \u001b[36m0:00:01\u001b[0m\n",
      "\u001b[?25hInstalling collected packages: python-dateutil, cronsim, aiocron\n",
      "  Attempting uninstall: python-dateutil\n",
      "    Found existing installation: python-dateutil 2.8.2\n",
      "    Uninstalling python-dateutil-2.8.2:\n",
      "      Successfully uninstalled python-dateutil-2.8.2\n",
      "Successfully installed aiocron-2.1 cronsim-2.6 python-dateutil-2.9.0.post0\n",
      "\u001b[33mWARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv\u001b[0m\u001b[33m\n",
      "\u001b[0m\n",
      "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m A new release of pip is available: \u001b[0m\u001b[31;49m23.2.1\u001b[0m\u001b[39;49m -> \u001b[0m\u001b[32;49m25.0.1\u001b[0m\n",
      "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m To update, run: \u001b[0m\u001b[32;49mpip install --upgrade pip\u001b[0m\n"
     ]
    }
   ],
   "source": [
    "!pip install aiocron"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "id": "59ce41e3",
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "from typing import Any, AsyncGenerator, Awaitable, Callable, Dict, Optional\n",
    "from aiocron import crontab\n",
    "from pydantic import BaseModel, Field\n",
    "from pytz import BaseTzInfo\n",
    "\n",
    "from metagpt.actions.action import Action\n",
    "from metagpt.logs import logger\n",
    "from metagpt.roles import Role\n",
    "from metagpt.schema import Message\n",
    "\n",
    "# fix SubscriptionRunner not fully defined\n",
    "from metagpt.environment import Environment as _  # noqa: F401"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "06b37c1e",
   "metadata": {},
   "source": [
    "## 订阅模块，可以from metagpt.subscription import SubscriptionRunner导入，这里贴上代码供参考\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "6fa60038",
   "metadata": {},
   "outputs": [],
   "source": [
    "class SubscriptionRunner(BaseModel):\n",
    "    \"\"\"A simple wrapper to manage subscription tasks for different roles using asyncio.\n",
    "    Example:\n",
    "        >>> import asyncio\n",
    "        >>> from metagpt.subscription import SubscriptionRunner\n",
    "        >>> from metagpt.roles import Searcher\n",
    "        >>> from metagpt.schema import Message\n",
    "        >>> async def trigger():\n",
    "        ...     while True:\n",
    "        ...         yield Message(\"the latest news about OpenAI\")\n",
    "        ...         await asyncio.sleep(3600 * 24)\n",
    "        >>> async def callback(msg: Message):\n",
    "        ...     print(msg.content)\n",
    "        >>> async def main():\n",
    "        ...     pb = SubscriptionRunner()\n",
    "        ...     await pb.subscribe(Searcher(), trigger(), callback)\n",
    "        ...     await pb.run()\n",
    "        >>> asyncio.run(main())\n",
    "    \"\"\"\n",
    "\n",
    "    tasks: Dict[Role, asyncio.Task] = Field(default_factory=dict)\n",
    "\n",
    "    class Config:\n",
    "        arbitrary_types_allowed = True\n",
    "\n",
    "    async def subscribe(\n",
    "        self,\n",
    "        role: Role,\n",
    "        trigger: AsyncGenerator[Message, None],\n",
    "        callback: Callable[\n",
    "            [\n",
    "                Message,\n",
    "            ],\n",
    "            Awaitable[None],\n",
    "        ],\n",
    "    ):\n",
    "        \"\"\"Subscribes a role to a trigger and sets up a callback to be called with the role's response.\n",
    "        Args:\n",
    "            role: The role to subscribe.\n",
    "            trigger: An asynchronous generator that yields Messages to be processed by the role.\n",
    "            callback: An asynchronous function to be called with the response from the role.\n",
    "        \"\"\"\n",
    "        loop = asyncio.get_running_loop()\n",
    "\n",
    "        async def _start_role():\n",
    "            async for msg in trigger:\n",
    "                resp = await role.run(msg)\n",
    "                await callback(resp)\n",
    "\n",
    "        self.tasks[role] = loop.create_task(_start_role(), name=f\"Subscription-{role}\")\n",
    "\n",
    "    async def unsubscribe(self, role: Role):\n",
    "        \"\"\"Unsubscribes a role from its trigger and cancels the associated task.\n",
    "        Args:\n",
    "            role: The role to unsubscribe.\n",
    "        \"\"\"\n",
    "        task = self.tasks.pop(role)\n",
    "        task.cancel()\n",
    "\n",
    "    async def run(self, raise_exception: bool = True):\n",
    "        \"\"\"Runs all subscribed tasks and handles their completion or exception.\n",
    "        Args:\n",
    "            raise_exception: _description_. Defaults to True.\n",
    "        Raises:\n",
    "            task.exception: _description_\n",
    "        \"\"\"\n",
    "        i=0\n",
    "        while True:\n",
    "            i+=1\n",
    "            for role, task in self.tasks.items():\n",
    "                i=0\n",
    "                if task.done():\n",
    "                    if task.exception():\n",
    "                        if raise_exception:\n",
    "                            raise task.exception()\n",
    "                        logger.opt(exception=task.exception()).error(\n",
    "                            f\"Task {task.get_name()} run error\"\n",
    "                        )\n",
    "                    else:\n",
    "                        logger.warning(\n",
    "                            f\"Task {task.get_name()} has completed. \"\n",
    "                            \"If this is unexpected behavior, please check the trigger function.\"\n",
    "                        )\n",
    "                    self.tasks.pop(role)\n",
    "                    break\n",
    "            else:\n",
    "                await asyncio.sleep(1)\n",
    "            if i>0:\n",
    "                break"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0f0b4c73",
   "metadata": {},
   "source": [
    "# Actions 的实现\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "id": "d76d909b",
   "metadata": {},
   "outputs": [],
   "source": [
    "RENDING_ANALYSIS_PROMPT = \"\"\"# Requirements\n",
    "You are a GitHub Trending Analyst, aiming to provide users with insightful and personalized recommendations based on the latest\n",
    "GitHub Trends. Based on the context, fill in the following missing information, generate engaging and informative titles, \n",
    "ensuring users discover repositories aligned with their interests.\n",
    "\n",
    "# The title about Today's GitHub Trending\n",
    "## Today's Trends: Uncover the Hottest GitHub Projects Today! Explore the trending programming languages and discover key domains capturing developers' attention. From ** to **, witness the top projects like never before.\n",
    "## The Trends Categories: Dive into Today's GitHub Trending Domains! Explore featured projects in domains such as ** and **. Get a quick overview of each project, including programming languages, stars, and more.\n",
    "## Highlights of the List: Spotlight noteworthy projects on GitHub Trending, including new tools, innovative projects, and rapidly gaining popularity, focusing on delivering distinctive and attention-grabbing content for users.\n",
    "---\n",
    "# Format Example\n",
    "\n",
    "\n",
    "# [Title]\n",
    "\n",
    "## Today's Trends\n",
    "Today, ** and ** continue to dominate as the most popular programming languages. Key areas of interest include **, ** and **.\n",
    "The top popular projects are Project1 and Project2.\n",
    "\n",
    "## The Trends Categories\n",
    "1. Generative AI\n",
    "    - [Project1](https://github/xx/project1): [detail of the project, such as star total and today, language, ...]\n",
    "    - [Project2](https://github/xx/project2): ...\n",
    "...\n",
    "\n",
    "## Highlights of the List\n",
    "1. [Project1](https://github/xx/project1): [provide specific reasons why this project is recommended].\n",
    "...\n",
    "\n",
    "---\n",
    "# Github Trending\n",
    "{trending}\n",
    "\"\"\"\n",
    "\n",
    "\n",
    "class CrawlOSSTrending(Action):\n",
    "    async def run(self, url: str = \"https://github.com/trending\"):\n",
    "        async with aiohttp.ClientSession() as client:\n",
    "            async with client.get(url, proxy=Config.default().proxy) as response:\n",
    "                response.raise_for_status()\n",
    "                html = await response.text()\n",
    "\n",
    "        soup = BeautifulSoup(html, \"html.parser\")\n",
    "\n",
    "        repositories = []\n",
    "\n",
    "        for article in soup.select(\"article.Box-row\"):\n",
    "            repo_info = {}\n",
    "\n",
    "            repo_info[\"name\"] = (\n",
    "                article.select_one(\"h2 a\")\n",
    "                .text.strip()\n",
    "                .replace(\"\\n\", \"\")\n",
    "                .replace(\" \", \"\")\n",
    "            )\n",
    "            repo_info[\"url\"] = (\n",
    "                \"https://github.com\" + article.select_one(\"h2 a\")[\"href\"].strip()\n",
    "            )\n",
    "\n",
    "            # Description\n",
    "            description_element = article.select_one(\"p\")\n",
    "            repo_info[\"description\"] = (\n",
    "                description_element.text.strip() if description_element else None\n",
    "            )\n",
    "\n",
    "            # Language\n",
    "            language_element = article.select_one(\n",
    "                'span[itemprop=\"programmingLanguage\"]'\n",
    "            )\n",
    "            repo_info[\"language\"] = (\n",
    "                language_element.text.strip() if language_element else None\n",
    "            )\n",
    "\n",
    "            # Stars and Forks\n",
    "            stars_element = article.select(\"a.Link--muted\")[0]\n",
    "            forks_element = article.select(\"a.Link--muted\")[1]\n",
    "            repo_info[\"stars\"] = stars_element.text.strip()\n",
    "            repo_info[\"forks\"] = forks_element.text.strip()\n",
    "\n",
    "            # Today's Stars\n",
    "            today_stars_element = article.select_one(\n",
    "                \"span.d-inline-block.float-sm-right\"\n",
    "            )\n",
    "            repo_info[\"today_stars\"] = (\n",
    "                today_stars_element.text.strip() if today_stars_element else None\n",
    "            )\n",
    "\n",
    "            repositories.append(repo_info)\n",
    "\n",
    "        return repositories\n",
    "\n",
    "\n",
    "class AnalysisOSSTrending(Action):\n",
    "    async def run(self, trending: Any):\n",
    "        return await self._aask(TRENDING_ANALYSIS_PROMPT.format(trending=trending))\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "id": "f20661c8",
   "metadata": {},
   "outputs": [],
   "source": [
    "\n",
    "# Role实现\n",
    "# 对于V0.7 以上的版本，需要把老版本的\n",
    "# self._init_actions 改为self.set_actions\n",
    "class OssWatcher(Role):\n",
    "    def __init__(\n",
    "        self,\n",
    "        name=\"Codey\",\n",
    "        profile=\"OssWatcher\",\n",
    "        goal=\"Generate an insightful GitHub Trending analysis report.\",\n",
    "        constraints=\"Only analyze based on the provided GitHub Trending data.\",\n",
    "    ):\n",
    "        super().__init__(name=name, profile=profile, goal=goal, constraints=constraints)\n",
    "        self.set_actions([CrawlOSSTrending, AnalysisOSSTrending])\n",
    "        self._set_react_mode(react_mode=\"by_order\")\n",
    "\n",
    "    async def _act(self) -> Message:\n",
    "        logger.info(f\"{self._setting}: ready to {self.rc.todo}\")\n",
    "        # By choosing the Action by order under the hood\n",
    "        # todo will be first SimpleWriteCode() then SimpleRunCode()\n",
    "        todo = self.rc.todo\n",
    "\n",
    "        msg = self.get_memories(k=1)[0]  # find the most k recent messages\n",
    "        result = await todo.run(msg.content)\n",
    "\n",
    "        msg = Message(content=str(result), role=self.profile, cause_by=type(todo))\n",
    "        self.rc.memory.add(msg)\n",
    "        return msg\n",
    "\n",
    "async def wxpusher_callback(msg: Message):\n",
    "    print(msg.content)\n",
    "\n",
    "\n",
    "async def trigger():\n",
    "    # 这里设置了只触发五次，也可以用while True 永远执行下去\n",
    "    for i in range(5):\n",
    "        yield Message(\"the latest news about OpenAI\")\n",
    "        await asyncio.sleep(5)\n",
    "        #  每隔五秒钟执行一次。\n",
    "        # 也可以设置为每隔3600 * 24 秒执行一次\n",
    "    \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c02a9f01",
   "metadata": {},
   "outputs": [],
   "source": [
    "# 运行入口，\n",
    "async def main():\n",
    "    callbacks = []\n",
    "    if not callbacks:\n",
    "        async def _print(msg: Message):\n",
    "            print(msg.content)\n",
    "        callbacks.append(_print)\n",
    "\n",
    "    # callback\n",
    "    async def callback(msg):\n",
    "        await asyncio.gather(*(call(msg) for call in callbacks))\n",
    "\n",
    "    runner = SubscriptionRunner()\n",
    "    await runner.subscribe(OssWatcher(), trigger(), callback)\n",
    "    await runner.run()\n",
    "    \n",
    "await main()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "55ff1d81",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.11"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
